在 GPU 加速中，我们必须摒弃“计算优先”的思维。现代性能主要由 内存管理决定：即主机（CPU）与设备（GPU）之间数据分配、同步和优化的协调。
1. 内存与计算的差距
尽管 GPU 的算术吞吐量（$TFLOPS$）已急剧上升，但内存带宽（$GB/s$）的增长却慢得多。这造成了一个鸿沟，执行单元经常处于‘饥饿’状态，等待来自显存的数据到达。因此， GPU 编程往往就是内存编程。
2. 屋顶线模型
该模型直观展示了 算术强度 （FLOPs/Byte）与性能之间的关系。应用程序通常分为两类：
- 内存受限： 受带宽限制（陡峭斜坡）。
- 计算受限： 受峰值 TFLOPS 限制（水平天花板）。
3. 数据移动的代价
性能的主要瓶颈很少是数学运算本身；而是通过 PCIe 总线或从高带宽内存（HBM）移动一个字节所带来的延迟和能耗成本。高性能代码更注重数据驻留，尽量减少主机与设备之间的数据传输。
main.py
TERMINALbash — 80x24
> Ready. Click "Run" to execute.
>
QUESTION 1
What is the primary cause of a GPU kernel being 'memory-bound'?
The clock speed of the GPU cores is too slow.
The rate of data delivery is slower than the rate of arithmetic execution.
There are too many threads running in parallel.
The CPU is faster than the GPU.
✅ Correct!
Correct! When data cannot be fed to execution units fast enough to keep them busy, the kernel is limited by memory bandwidth.❌ Incorrect
Memory-bound refers specifically to the bandwidth bottleneck, not core clock speeds.QUESTION 2
In the context of GPU programming, what does 'Memory Management' involve?
Only allocating variables on the CPU stack.
Controlling allocation, synchronization, and optimization of data transfer between host and device.
Optimizing the cache size of the L1 controller.
Manually cleaning the GPU registers after every kernel call.
✅ Correct!
Correct. It is the strategic orchestration of data across the entire hardware hierarchy.❌ Incorrect
Memory management in HIP/ROCm encompasses the movement and lifecycle of data between Host and Device.QUESTION 3
Which axis of the Roofline Model represents 'Arithmetic Intensity'?
Vertical Axis (Y)
Horizontal Axis (X)
The slope of the line.
The area under the curve.
✅ Correct!
Correct. The X-axis measures FLOPs per Byte, determining where an application sits relative to the bandwidth wall.❌ Incorrect
The Y-axis represents performance (GFLOPS); the X-axis represents intensity.QUESTION 4
Why is redundant host-device transfer considered a 'performance tax'?
It consumes GPU registers.
Latency and energy consumption of moving data across PCIe is significantly higher than instruction execution.
It increases the floating-point precision error.
It causes the GPU to overheat instantly.
✅ Correct!
Correct. Data movement is often the most expensive operation in terms of both time and power.❌ Incorrect
Data movement doesn't affect math precision; it affects performance and power efficiency.QUESTION 5
If a researcher's kernel spends 95% of its time 'stalled,' what is the most likely culprit?
The math instructions are too complex.
Inefficient orchestration of data residence causing the GPU to wait for data.
The GPU has too much VRAM.
The kernel was written in C++ instead of Python.
✅ Correct!
Correct. Stalls usually indicate the compute units are idle while waiting for high-latency memory transactions.❌ Incorrect
Complex math would make a kernel compute-bound, not necessarily cause 95% idle stalls.Case Study: The Climate Simulation Bottleneck
Optimizing a Fluid Dynamics Kernel
A research team is running a massive climate simulation. Their HIP kernel calculates fluid dynamics at high TFLOPS theoretically, but Profiling shows the GPU spends 95% of its time stalled. The team currently transfers data from Host to Device at every time-step.
Q
Why does transferring data at every time-step likely cause the 95% stall?
Solution:
The PCIe bottleneck: The time taken to move data between Host RAM and Device VRAM via the interconnect is orders of magnitude slower than the kernel execution, forcing the GPU to wait (stall) for the next set of data.
The PCIe bottleneck: The time taken to move data between Host RAM and Device VRAM via the interconnect is orders of magnitude slower than the kernel execution, forcing the GPU to wait (stall) for the next set of data.
Q
Based on the axiom 'GPU programming is memory programming,' what should the team's first optimization step be?
Solution:
Strategic orchestration of data residence: The team should keep data on the GPU across multiple time-steps and only transfer results back to the host when necessary, minimizing 'redundant' transfers.
Strategic orchestration of data residence: The team should keep data on the GPU across multiple time-steps and only transfer results back to the host when necessary, minimizing 'redundant' transfers.